Skip to content

Add Krisp VIVA Turn as a user turn-stop strategy#178

Open
weiz9 wants to merge 2 commits into
mainfrom
pr/wz/krisp-viva-turn
Open

Add Krisp VIVA Turn as a user turn-stop strategy#178
weiz9 wants to merge 2 commits into
mainfrom
pr/wz/krisp-viva-turn

Conversation

@weiz9

@weiz9 weiz9 commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds Krisp VIVA Turn (pipecat's KrispVivaTurn, VIVA SDK turn detection v3 / Tt API) as a selectable turn_stop_strategy in the cascade pipeline, alongside the existing LocalSmartTurnAnalyzerV3. Unlike Smart Turn (which analyzes audio in batches when VAD detects a pause), KrispVivaTurn processes audio frame-by-frame in real time, using the VAD's speech flag as an external signal.

Changes

  • turn_config.py — new krisp_viva_turn branch in create_turn_stop_strategy(). Forwards threshold / frame_duration_ms to KrispTurnParams and model_path / api_key / sample_rate to the KrispVivaTurn constructor (all fall back to env vars). The pipecat module is imported lazily because it hard-requires the proprietary krisp_audio SDK — environments without it are unaffected until the strategy is selected.
  • config.py + .env.example — document the new strategy and required env vars (KRISP_VIVA_TURN_MODEL_PATH, KRISP_VIVA_API_KEY).
  • test_turn_config.py — 4 new tests covering the branch (happy path, case-insensitivity, tuning-param routing, constructor-param routing) via an injected fake krisp module.
  • Bumped simulation_version 2.0.1 → 2.0.2 (affects benchmark outputs).

Enabling it

Krisp is a licensed SDK, not a PyPI package, so it is intentionally not in pyproject.toml. To run this strategy you provision it at runtime:

  1. Download from the Krisp developer portal (developers.krisp.ai): the VIVA UAR Python SDK zip and the Turn-Taking models zip. Unzip both.
  2. Install the wheel matching your interpreter's Python version and architecture from the SDK's dist/ folder — e.g. macOS x86_64 / Py3.11:
    uv pip install .../krisp-viva-uar-python-sdk-<ver>/dist/krisp_audio-<ver>-cp311-cp311-macosx_11_0_x86_64.whl
    (Pick arm64 vs x86_64 to match python -c "import platform; print(platform.machine())".)
  3. Point env vars at the model + your license key, then select the strategy:
    export KRISP_VIVA_TURN_MODEL_PATH=/path/to/krisp-viva-tt-models/krisp-viva-tp-v3.kef
    export KRISP_VIVA_API_KEY=<license key>
    eva --domain airline --model.llm gpt-5-mini --model.turn-stop-strategy krisp_viva_turn

Optional tuning via EVA_MODEL__TURN_STOP_STRATEGY_PARAMSthreshold (default 0.5) and frame_duration_ms (default 20; one of 10/15/20/30/32).

Network: the SDK validates its license against sdkapi.krisp.ai and reports usage to analytics.krisp.ai. Both must be reachable; a WAF/proxy that blocks them will fail license validation. Eval keys are time-limited and metered per minute.

Deployment (toolkit / Docker)

The runtime image is linux/amd64, so it needs the Linux wheel from the same SDK download — krisp_audio-<ver>-cp311-cp311-linux_x86_64.whl (ships alongside the macOS wheels). The wheel + .kef model should be baked into the image from an internal artifact store (not committed to git), with KRISP_VIVA_API_KEY injected as a runtime secret and egress to the two Krisp hosts allowlisted. Tracked as follow-up.

Testing

  • 27/27 test_turn_config.py unit tests pass; ruff + version-bump checks pass; no new mypy errors.
  • Live end-to-end validated (macOS, cascade pipeline): a full multi-turn airline conversation ran with turn_stop_strategy=krisp_viva_turn; the SDK initialized, loaded the model, and produced 90 real end-of-turn predictions (model probabilities 51–99%, end-of-turn latencies ~0.1 ms–1.5 s, mostly ~100–300 ms). Turn-taking was driven entirely by Krisp.
  • Also confirmed the lazy-import boundary: when krisp_audio is absent, selecting the strategy fails with a clean, actionable error and no other strategy is affected.

weiz9 added 2 commits July 8, 2026 09:58
Wire pipecat's KrispVivaTurn (VIVA SDK turn detection v3) into the cascade
pipeline as a selectable turn_stop_strategy alongside the existing
LocalSmartTurnAnalyzerV3. The analyzer is imported lazily because it depends
on the proprietary krisp_audio SDK plus a .kef model and license key, so
environments without the SDK are unaffected until the strategy is selected.

- turn_config: add 'krisp_viva_turn' branch forwarding threshold/
  frame_duration_ms to KrispTurnParams and model_path/api_key/sample_rate
  to the KrispVivaTurn constructor (both fall back to env vars)
- config + .env.example: document the strategy and required env vars
  (KRISP_VIVA_TURN_MODEL_PATH, KRISP_VIVA_API_KEY)
- tests: cover the new branch via an injected fake krisp module
- bump simulation_version 2.0.1 -> 2.0.2
@weiz9
weiz9 marked this pull request as ready for review July 23, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant